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1.  Introduction 


In  previous  work  we  performed  a  comparative  survey  of  several  different  methods  of  designing 
a  feedback  control  for  nonlinear  systems  [1].  Since  finding  the  exact  optimal  feedback  control  is, 
in  general,  impossible  for  the  nonlinear  case,  the  methods  used  various  approaches  to  finding  an 
effective  suboptimal  control.  In  this  paper  we  will  extend  one  of  the  nonlinear  feedback  control 
methods  to  treat  problems  involving  tracking  control  and  state  estimation,  which  would  allow  it  to 
be  used  in  a  wide  class  of  real  applications. 

Both  tracking  and  estimation,  like  the  simple  optimal  feedback  control  problem,  have  been 
widely  studied  for  linear  systems,  and  methods  of  formulation  of  the  control  and  estimator  are  well 
known  (for  example,  see  [2,  3]  and  the  references  they  contain).  In  the  linear  case  the  tracking 
problem  is  solved  in  two  parts:  a  feedback  control  determined  by  the  algebraic  Riccati  equation 
(ARE),  and  a  time-dependent  tracking  variable  found  by  solving  an  ordinary  differential  equation 
(incorporating  the  desired  trajectory)  backwards  from  a  stable  final  time.  The  state  estimation 
problem  for  linear  systems  involves  the  formulation  of  a  state  estimator  system  incorporating  the 
observed  measurement  of  the  actual  state  and  a  gain  matrix  found  through  a  second  Riccati  equation. 

To  construct  observers  and  tracking  controls  for  the  nonlinear  case  we  first  hoped  to  extend 
the  two-term  power  series  control  of  Garrard  [4],  given  its  simplicity  and  effectiveness  in  the  types 
of  problems  studied  in  [1],  However,  since  it  is  based  directly  on  the  Hamilton- Jacobi-Bellman 
equation  for  optimal  feedback  control,  and  the  linear  tracking  control  in  particular  is  derived  from 
the  Hamiltonian  state/costate  formulation  of  the  control  problem,  combining  the  two  proved  difficult. 
Instead,  our  approaches  are  based  on  the  state-dependent  Riccati  equation  (SDRE)  [5,  6,  7,  8,  9], 
which  is  also  very  simple  to  calculate,  though  not  quite  as  effective  as  the  two-term  power  series 
method  at  controlling  some  chosen  test  problems  in  [1],  However,  as  shown  in  this  paper,  the  SDRE 
method  is  more  readily  adaptable  to  the  nonlinear  tracking  and  nonlinear  state  estimation  problems, 
since  it  is  closely  related  to  the  ARE-based  method  used  to  find  those  controls  in  the  linear  problem. 

While  there  is  a  large  amount  of  literature  available  on  state  estimation  for  nonlinear  systems,  a 
literature  search  revealed  very  little  material  on  tracking  control  for  nonlinear  systems.  The  tracking 
control  technique  discussed  in  this  paper  was  developed  directly  out  of  the  combination  of  the  SDRE 
for  nonlinear  control  with  the  linear  method  for  tracking  control.  The  method  of  state  estimation 
we  will  describe  below  is  closely  related  to  work  by  Thau  and  others  [10,  11,  12,  13]  on  nonlinear 
estimators  which  are  themselves  extensions  of  the  linear  state  estimation  formula. 

There  are  other  very  different  methods  for  state  estimation  described  in  the  literature  which  we 
decided  not  to  use,  mostly  because  they  are  difficult  to  implement,  though  theoretically  solid.  A 
large  effort  has  been  made  with  types  of  methods  that  use  a  nonlinear  transformation  to  change  the 
subject  system  into  a  very  specific  observer  form,  mostly  using  many  Lie  derivatives  of  the  nonlinear 
functions  from  the  problem  measurement  and  dynamics  [14,  15,  16,  17,  18,  19,  20].  Other  methods 
use  Lie  derivatives  as  well  but  not  in  a  transformation  to  a  simpler  system  form  [21,  22],  Also,  there 
are  methods  using  a  linearization  about  a  family  of  constant  operating  points  of  the  system  [23], 
variable-structure  system  techniques  [24],  and  Lyapunov  theory  [25].  A  number  of  these  methods 
are  compared  in  the  survey  paper  of  Walcott,  Corless  and  Zak  [26].  These  methods  prove  difficult 
to  use  in  many  applications,  especially  problems  with  complicated  nonlinearities  in  the  dynamics 
or  measurement.  More  specifically,  the  nonlinear  transformation  is  often  very  hard  to  find  or  may 
not  even  exist,  higher-order  Lie  derivatives  are  often  hard  to  calculate,  the  linearization  around 
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the  family  of  solutions  may  be  very  difficult,  and  finding  the  appropriate  Lyapunov  function  may  be 
quite  challenging.  There  may  be  restrictions  on  the  types  of  problems  for  which  various  methods  can 
guarantee  results  theoretically,  or  even  be  used  at  all.  The  state-dependent  Riccati  equation  based 
method  that  we  develop  involves  some  approximations  and  assumptions,  but  it  is  straightforward  to 
implement  and  is  applicable  to  a  certain  class  of  nonlinear  systems  that  are  important  in  practice. 

In  Section  2  of  the  paper  we  will  give  a  brief  description  of  the  state-dependent  Riccati  equation 
in  the  context  of  simple  feedback  control  of  nonlinear  systems.  Section  3  will  focus  on  our  formulation 
of  the  feedback  tracking  control,  and  Section  4  on  the  state  estimator,  for  the  nonlinear  case.  The 
controls  will  be  implemented  on  two  test  problems  in  Section  5,  and  the  results  will  be  discussed  and 
compared  against  the  linear  tracking  control  and  linear  and  partially  nonlinear  estimators.  Finally, 
overall  conclusions  will  be  given  in  Section  6. 

2.  The  State-Dependent  Riccati  Equation 

The  state-dependent  Riccati  equation  (SDRE)  is  one  method  for  obtaining  a  feedback  control  for 
nonlinear  systems,  and  is  described  in  detail  by  Cloutier,  D’Souza  and  Mracek  in  [9].  It  is  simply  an 
extension  of  the  constant-valued  algebraic  Riccati  equation  used  to  find  the  optimal  feedback  control 
in  the  linear  quadratic  regulator  problem,  which  is  well  established  for  finite  and  infinite  dimensional 
problems  (see  e.g.  [27,  28]).  There  are  also  stable  and  robust  algorithms  for  solving  the  algebraic 
Riccati  equation  available  in  the  literature. 

Consider  a  system  of  the  type 


x(t)  =  f{x(t))  +  Bu(x(t)) 

x(0)  =  x0, 


(1) 


which  is  nonlinear  in  the  state  ie!lc  Rm  and  is  linear  in  the  control  a  :  Q  — >  Rk.  and  a  quadratic 
cost  functional 


J(x o,  u)  =  —  J  (xTQx  +  uTRuj  dt 


with  given  constant- valued  weighting  matrices  Q  and  R  of  appropriate  dimensions.  The  optimal 
control  problem  is  to  find  a  state  feedback  control  u*{x*)  which  minimizes  the  cost  for  all  possible 
initial  conditions  xo. 

We  begin  with  the  Hamiltonian  for  this  problem,  which  is  defined  as 
'H  (x,  u,p)  =  ^ xtQx  +  ^ uT Ru  +  pT  ( f(x )  +  Bu) . 

From  the  Hamiltonian  the  necessary  conditions  for  the  optimal  control  are  given  by 


fYl J 

x*  =  —  =  f(x*)  +  Bu* 

■  *  dR  dfT 

P  =  —^  = -Qx*  - -±^-(x*)p* 

d  H  ^  *  wr  * 

=  —  =  Ru*  +  BTp*. 
ou 


(2) 


0 


3 


(3) 

(4) 


From  equation  (4),  the  control  is  given  in  terms  of  the  costate  variable,  p*(t ),  by  u*(t)  =  —R~1BTp*(t). 

We  will  seek  a  costate  of  the  formp*(f)  =  U  (x*  (t))  x* (t) .  First,  taking  the  derivative  of  p* 
yields  p*  =  II  (a;*)  x*  +  DtH  (x*)  x*.  Then  we  substitute  into  this  equation  the  formulas  for  x*  and 
p*  determined  by  the  necessary  conditions  (2)  and  (3),  and  the  formulas  for  u*  and  p*  given  above, 
resulting  in 


n  (x*)  [f{x*)  -  br~1btb  (x*)  x*]  +  a n  (®*)  x*  =  -Qx*  -  ^-{x*)u  (x*)  x *, 


dx 


which  can  be  rewritten  as 


n  (x*)  fix*)  +  -4-  (x*)U  (x*)  x*  -  n  (x*)  BR~1BtU  {x*)  x*  +  Qx *  +  DtU  (x*)  x *  =  0.  (5) 

ox 

The  term  DtII(x)  is  a  somewhat  misleading  notation;  it  is  the  total  time  derivative  of  II(x(i))  given 
by 


m  /in 

Dtn(x)  = 

k= i OXk 

and  thus  has  no  meaning  except  when  evaluated  along  a  state  trajectory  x(t)  so  that  x  can  have 
some  value. 

For  the  simpler  linear  problem,  where  the  dynamics  are  f(x)  =  Aqx,  x*  can  be  factored  out  of 
each  term  in  equation  (5)  so  that  it  becomes 

IL40  +  A^U-UBR~1BtU  +  Q  =  0.  (6) 

Equation  (6)  is  now  the  standard  algebraic  Riccati  equation  with  a  constant-valued  solution  matrix 
II.  The  resulting  optimal  feedback  control  is  given  by  u*{x*)  =  —R~lBTIix* . 

The  SDRE  method  involves  mimicking  the  above  use  of  the  Riccati  equation  by  rewriting  the 
nonlinear  function  of  x  in  (1)  as  f{x)  =  A(x)x.  Note  that  the  choice  of  the  state-dependent  matrix 
function  A  is  not  unique,  and  different  controls  will  result  from  different  choices  of  A.  With  / 
rewritten  in  this  way  equation  (5)  becomes 


n  (x*)  A(x*)x*  +  at(x*)  n  (x*)  x*  +  Y 


*  f  dAi~ j  * 


Xa 


i= l 

>—l  r>Ti 


{x*)\  n(x!,!)x!,: 


V  dx 

-n  (s*)  BR~lBT n  (x*)  x*  +  Qx*  +  DtU  {. X *)  x* 


=  0, 


which  can  then  be  rewritten  as 


n  (x*)  a(x*)  +  a1  (x*)n  (x*)  -  n  (x*)  br ~lb1  n  {x*)  +  Q 


+  Y,xi 

i= 1 


dA 


1  —tm,i  /  * ' 


dx 


{x*)\  n(x*)  +DtU(x*)  \x*  =  0, 


where  the  xl-column  derivatives  are  given  by 
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(7) 


9A\ ~^m,i 
dx 


/  dAu/dxi 
V  dAmi/dx  1 


dAn/dxm  \ 
dAmi/  dxm  J 


This  equation  can  be  divided  into  the  state-dependent  Riccati  equation 


II(a:)T(a:)  +  AT  (x)H(x)  —  H(x)BR  1BTH(x)  +  Q  =  0,  (8) 

and  two  extra  terms 

E xi  n (*)  +  An (*)  =  o.  (9) 

In  the  SDRE  method  one  assumes  the  extra  terms  remain  small  and  ignores  them  (thus,  among 
other  things,  removing  the  necessity  of  specifying  a  trajectory  x(t)  so  that  D/Il(x)  can  be  evaluated 
in  the  equation) .  This  assumption  creates  a  suboptimal  feedback  control  based  on  the  solution  to 
the  Riccati  equation  (8)  and  given  by 


u(x)  =  —R  1BtU(x)x.  (10) 

In  almost  all  problems  the  SDRE  solution  n(x)  of  (8)  is  much  more  difficult  to  find  than  for  the 
constant-valued  case  (6).  Wernli  and  Cook  [7]  propose  a  method  for  solving  the  Riccati  equation 
(8)  using  a  power  series  expansion,  which  we  will  follow.  We  begin  by  splitting  A  into  a  constant 
part  and  a  state-dependent  part  as  A(x)  =  A  +  eAA(x),  where  e  is  a  temporary  variable  used  for 
the  expansion  that  will  be  set  to  1  later.  We  next  write  II  as  a  power  series  in  e,  as 

OO 

n  (x,e)  =  EenA(A  (11) 

n= 0 

where  II  as  well  as  each  Ln  is  symmetric.  Substituting  these  expansions  into  the  state-dependent 
Riccati  equation  (8),  we  find 


E  £nLn{x )  (A)  +  eA A(x))  +  (Rq  +  eA AT(x)^j  I  E  £nLn{x) 


Vn=0 


\n=0 


-  E  enLn(x)  BR~1Bt  E  e"A,(*)  +  Q  =  0. 

\n= 0  J  \n= 0  / 

We  then  match  terms  with  the  same  powers  of  e  to  obtain  the  following  set  of  equations  to  determine 
Rn- 


LqAq  +  Alq Lq  —  LqBR  1BtLo  +  Q  = 

0 

(12) 

A  (a  -  br~1btl0 )  + 

(a%  -  LqBR~1Bt^  Li  +  L0AA  +  A ATL0  = 

0 

(13) 

Ln  ^A  —  BR  +  (Aq 

—  LqBR  ^ B^^j  Ln  -j-  Ln—\AA  +  ATTLn_i 

n—  1 

-  E  (LkBR~lBTLn_k)  = 

k=l 

0. 

(14) 
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Equation  (12)  is  the  standard  Riccati  equation  for  the  linear  part  of  the  system,  Aq  ,  and  is  easily 
solved,  but  equations  (13)  and  (14)  may  be  tedious  to  solve  if  the  function  AA(x)  is  complicated. 
However,  for  a  certain  class  of  nonlinear  functions,  the  higher-order  terms  in  the  SDRE  expansion 
are  much  easier  to  consider.  Specifically,  we  consider  those  problems  in  which  A  A  has  the  same 
function  of  x  in  all  of  its  entries,  so  it  can  be  written  as  A A(x)  =  g(x)AAc  with  a  constant-valued 
matrix  A  Ac  (both  of  the  examples  in  Section  5  are  of  this  type).  By  defining  Ln(x)  =  gn(x)(Ln)c , 
where  ( Ln)c  is  a  constant-valued  matrix,  we  can  factor  out  gn(x)  from  (14)  to  obtain 

(4,)c  (A  -  BR-1BtL0 )  +  (Al  -  L0BR~1BT)  ( Ln)c  +  (f„-i)cA4c  +  A^(L„_1)C 

n—  1 

-  Y,  ({Lk)cBR-1BT{Ln_k)c)  =  0. 

k= 1 

The  above  equation  is  a  constant-valued  matrix  Lyapunov  equation,  for  which  stable  and  efficient 
algorithms  exist  in  the  literature,  so  this  class  of  SDRE  problems  can  be  solved  to  as  many  terms 
of  the  power  series  as  desired.  Once  as  many  Ln  terms  as  desired  have  been  found,  the  control  is 
obtained  by  substituting  them  back  into  (11)  and  (10)  with  e  set  to  1. 

Some  theoretical  results  are  available  for  the  state-dependent  Riccati  equation  and  the  subopti- 
mal  feedback  control  law  derived  from  it.  Wernli  and  Cook  [7]  show  the  existence  and  asymptotic 
stability  of  the  controlled  system,  given  certain  assumptions  on  the  properties  of  system.  Cloutier, 
D’Souza  and  Mracek  [9]  prove,  given  certain  other  assumptions,  properties  of  local  and  global  sta¬ 
bility,  robustness  and  suboptimality  for  the  control.  Hammett,  Hall  and  Ridgely  [29]  discuss  the 
issues  of  controllability  and  stabilizability  (properties  important  to  the  proofs  in  [9])  in  regards  to 
the  SDRE  method  of  controlling  nonlinear  systems. 

The  SDRE  method  can  be  calculated  very  quickly  for  the  case  described  above  with  A A(x)  = 
g(x)AAc ,  but  this  assumption  on  the  form  of  A  A  does  limit  the  problems  for  which  the  SDRE 
approach  is  most  useful.  There  is  also  the  drawback  that  this  method  of  control  ignores  the  extra 
terms  in  equation  (9)  in  setting  up  the  SDRE,  and  additionally  the  power  series  solution  is  only 
an  approximation  to  the  exact  solution  of  the  SDRE  (in  particular  it  is  less  accurate  farther  from 
the  power  series  expansion  point  of  the  origin).  These  approximations  lead  to  only  a  suboptimal 
feedback  control,  although  the  application  of  the  method  to  test  problems  in  [1]  shows  generally  very 
good  results. 

3.  Tracking  Control  for  Nonlinear  Systems 

In  this  section  we  will  derive  an  analogous  SDRE-based  solution  to  the  feedback  tracking  control 
problem  for  a  nonlinear  system  in  a  manner  similar  to  the  derivation  in  [3]  for  a  linear  system.  We 
focus  on  the  differential  equation  system 

x(t)  =  f(x(t))  +  Bu(x(t),t) 

<  X'(0)  =  Xq 

.  y{t)  =  Hx(t). 

This  is  the  same  as  in  the  previous  section  with  the  addition  of  the  variable  y,  which  is  the  signal 
we  want  to  follow  along  a  desired  trajectory.  We  take  y  as  a  linear  function  of  the  state  variables 
for  now.  The  cost  function  for  the  tracking  problem,  with  a  desired  trajectory  r(t),  is  given  by 
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i  r°°  t  \ 

j{x 0, u)  =  2  J0  vy  -  r)T(3(y  ~r)  +  utRu J  dt. 

To  prepare  for  the  application  of  the  SDRE,  we  rewrite  the  nonlinear  function  as  f(x)  =  A(x)x. 
With  this  notation,  the  Hamiltonian  is  given  by 

R(x,u,p)  =  Hx  —  r)TQ(Hx  —  r)  +  ^ uT Ru  +  pT  (A(x)x  +  Bu) . 

The  necessary  conditions  for  the  optimal  control  are 


x*  =  —  =  A(x*)x*  +  Bu*  (15) 

op 

P*  =  =  —HtQ(Hx*  -  r)  -  AT (x*)p*  -  f>*  (d^m’V))T P*  (16) 

i— 1  ^  ' 

ftnj 

0  =  —  =  Ru*  +  Btp\  (17) 

ou 

with  the  /1-column  derivatives  as  defined  in  equation  (7).  From  equation  (17),  the  control  is  given 
by  the  form  u*(t)  =  —R~1BTp*(t). 

We  seek  a  costate  of  the  form  p*(t)  =  n  (x*(t))  x*(t)  +  s(t)  (with  a  time-dependent  tracking 
variable  s  added  to  the  version  from  Section  2)  which  satisfies  the  necessary  conditions.  As  before, 
we  take  the  derivative  of  p*  to  obtain  p*  =  n  (x*)  x*  +  s  +  DtH  {x*)  x*.  Then  we  substitute  in  the 
formulas  for  x*  and  p*  from  the  necessary  conditions  (15)  and  (16),  and  the  above  formulas  for  u* 
and  p* ,  yielding 


n  {x*)  [a{x*)x*  -  BR-xBt  (n  (®*)  x*  +  a)]  +  8  +  Dtu  (x*)  x*  =  —HtQ(Hx*  -  r) 


*  f  dA\—^m^  . 


-At(x*)(U(x*)x*  +  s)-J2^  v  ^ 


(**)  (H (a:*)®*  +  s)  , 


which  can  be  rewritten  as 


n  (®*)  A(x*)  +  At(x*)U  (. X *)  -  n  (x*)  BRrlBTIi  {x*)  +  HtQh\  X*  +  [s  +  At(x*)s 
-n  (x*)  BR~1Bts  -  HTQr  +  f^x*  (n  (®*)  X*  +  a)  +  DtU  (®*)  x*  I  =  0. 

This  separates  the  equation  into  two  parts:  a  state-dependent  Riccati  equation  for  determining  Ii{x) 
with  which  to  find  the  feedback  control  gain,  and  an  ODE  for  determining  the  tracking  variable  s(t). 

In  the  non-tracking  feedback  control  problem  in  Section  2,  the  terms  involving  derivatives  of 
A  and  n  in  (9)  were  assumed  to  be  small  and  were  neglected,  thus  reducing  the  problem  down  to 
Riccati  equation  form.  This  neglecting  of  terms  is  what  makes  the  SDRE  method  only  a  suboptimal 
means  of  control.  However,  now  there  is  a  second  part  to  the  problem,  the  tracking  variable  equation 
given  by 
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m  ( dA  \ 1 

s  +  At{x)s  -  n  (x)  BR~lDTs  -  HTQr  +  ^  x* — Y* m ,l  (. x )  j  (II  (x)  x  +  s)  +  DtU  (x)  x  —  0.  (18) 

i  1  \  ox  J 

We  note  that  the  solution  to  this  equation  is  state-dependent  (i.e.,  s  should  actually  be  written  as 
s(t,x))  through  the  presence  of  A(x)  and  n(x)  even  without  the  derivative  terms,  so  by  keeping  the 
derivative  terms  in  the  equation  we  can  include  their  effects  in  the  control  design  without  drastically 
changing  the  nature  of  the  problem. 

Solving  the  tracking  equation  is  more  difficult  for  this  nonlinear  problem  than  for  the  linear 
case  described  in  [3].  In  the  linear  problem  the  ODE  is  solved,  offline,  backwards  from  an  assumed 
final  time  value  s(Tf)  =  0.  For  the  nonlinear  case,  s(t.  x)  must  still  be  computed  offline  (since 
the  value  of  s  at  t  =  0  is  unknown),  but  now  this  computation  must  include  the  dependence  of  the 
equation  on  x  through  the  now  state-dependent  A  and  II  as  well  as  the  derivative  terms  not  present 
in  the  linear  case.  We  do  this  by  solving  for  s  using  a  trajectory  xnom(t)  found  with  the  state 
equation  xnom  =  A(xnom)xnom  ~  BR~1BT(U(xnom)  xnom  +  s),  coupled  to  equation  (18).  Given 
the  initial  condition  xnom(0)  =  xo  and  the  final  condition  s(Tf,  xnom(Tf))  =  0,  this  results  in  a  two- 
point  boundary  value  problem  which  can  be  discretized  and  solved  with  a  finite  difference  method 
for  xnom(t)  and  s(t,xnom(t)).  Note  that  the  coupling  of  the  s  and  xnom  equations  provides  the 
trajectory  xnorn(t)  necessary  to  evaluate  the  Dt U(xnom)  term  in  (18). 

We  use  the  notation  xnom  to  clarify  that  this  is  a  nominal  trajectory  solved  offline,  before  actual 
implementation  of  the  feedback  control  on  the  system.  If  the  system  dynamics  are  known  precisely 
then  it  will  match  the  actual  trajectory  precisely.  However,  for  example,  in  Section  5  the  control 
is  applied  to  a  problem  with  random  noise  in  the  x  system  but  not  in  the  nominal  xnom  system, 
to  represent  a  problem  where  an  actual  physical  system  x  has  some  unpredictable  noise  which  the 
model  xnom  is  unable  to  include.  In  the  offline  calculation  of  s  coupled  with  xnom  in  the  TPBV 
problem,  differences  between  x  and  xnom  due  to  an  altered  initial  value  or  some  variation  or  noise 
will  result  in  an  inaccurate  s  since  it  is  found  using  an  inaccurate  xnom.  We  tacitly  assume  that  xnorn 
is  a  good  prediction  of  the  actual  state  behavior,  and  that  any  small  differences  will  not  drastically 
impact  the  effectiveness  of  the  control.  Another  area  of  concern  in  this  tracking  control  method  is 
the  fact,  discussed  in  Section  2,  that  since  we  are  finding  a  power  series  based  solution  to  the  SDRE, 
it  will  be  more  inaccurate  the  farther  the  state  is  from  the  expansion  point,  which  is  the  origin.  This 
is  particularly  important  for  the  tracking  problem,  since  we  specifically  want  the  state  to  follow  a 
certain  nonzero  trajectory. 

To  summarize,  the  control  formula 

u(x,  t)  =  —  R_1Bt  (n  (x)  x  +  s(t,x))  (19) 

is  found  by,  first,  solving  the  SDRE 

n  (x)  A(x)  +  at(x) n  (x)  -  n  (x)  br~1bt n  (®)  +  htqh  =  o  (20) 

for  n  (x)  in  the  manner  described  in  Section  2.  The  second  part  of  the  control  design  process  is 
then  to  find  the  tracking  variable  s(t,xnom)  from 

s 

< 

Xnom 


A  {xnom)s  +  H  (x 

—  YaL  1  ( Xnom)i  (  dx„om  y^nom 

A{Xnom)%nom  ~  BR  B  (II  (x 


°nom 
+m,i 


)BR~1B1  s  +  H1  Qr 

{p nom  ))  (n  ( Xnom )  XTl 
)  Xnom  T  s) 


T  s)  DtU  ( Xn0m )  Xr, 


(21) 
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in  a  two-point  boundary  value  problem  with  xnom(0 )  =  xo  and  s(Tf,  xnom(Tf))  =  0. 

For  a  nonlinear  tracking  signal  y(t)  =  H(x(t))x(t),  the  control  problem  becomes  somewhat  more 
complicated.  The  state-dependence  of  H  affects  the  control  formulation  in  a  similar  way  to  that  of 
A.  resulting  in  the  Riccati  and  tracking  variable  equations  in  (20)  and  (18)  being  expanded  into  the 
following  forms: 


n  [x)  A(x)  +  at{x) n  (x)  -  n  (®)  br~1bth  (®)  +  ht{x)qh{x)  =  o  (22) 


m  /flu  \T 

s  +  At(x)s  —  II  (x)  BR~lBts  —  HT(x)Qr  +  ^  X{  (  — ( x )  j  Q  (H  (x)  x  —  r ) 

i=  1  \  J 


i= 1 


dAi 


—ym,i 


dx 


(x)  )  (II  (x)  x  +  s)  +  Dt II  (x)  x  =  0. 


The  tracking  variable  equation  is  only  slightly  changed,  adding  one  more  term  and  making  H  state- 
dependent  in  another,  neither  of  which  strongly  affects  the  solution  process.  The  SDRE  contains 
the  added  state-dependence  of  H  in  the  HTQH  term,  which  can  have  more  important  consequences. 
If  H{x)  is  not  a  very  complicated  formula  as  was  discussed  for  A{x)  in  Section  2,  such  as  H(x)  = 
Ho  +  g(x)AHc ,  then  it  may  simply  add  terms  to  the  equivalent  of  the  higher-order  equations  (13) 
and  (14)  in  the  power  series  solution  of  equation  (22).  A  problem  arises  if  H(x)  has  no  constant¬ 
valued  part  //().  but  only  higher-order  parts.  This  will  cause  the  Q-based  term  to  vanish  from  the 
tracking  problem  equivalent  of  the  first  equation  in  the  power  series  solution  (12),  which  results  in 
only  a  trivial  solution  to  that  equation  and  causes  difficulties  in  the  higher-order  equations  as  well. 
This  makes  finding  a  usable  feedback  control  with  this  method  very  difficult  for  that  case.  However, 
the  linear  tracking  signal  is  sufficient  for  a  large  number  of  tracking  problems,  and  we  will  consider 
only  that  form  of  the  problem  for  now. 


4.  State  Estimation  for  Nonlinear  Systems 

The  method  we  chose  for  the  state  estimation  is  related  to  the  linear-system  case  [2,  3]  and  to 
previous  work  on  nonlinear  systems  in  the  literature  [10,  11,  12,  13].  We  consider  a  system  of  ODEs 
with  nonlinear  dynamics  and  a  nonlinear  measurement  of  the  form 

j  x(t)  =  f(x(t))  +  Bu(xe(t),t) 

\  z(t)  =  c{x{t)). 

The  control  for  a  tracking  problem  is  given  by  u(xe,t)  =  —R~1BT(Il(xe)xe  +  s(t,xnam))  as  dis¬ 
cussed  in  the  last  section  except  now  in  terms  of  the  estimated  state  xe. 

We  look  for  a  state  estimator  of  type  xe  =  fc(xe,t )  +  F(z,xe).  If  the  error  in  the  estimation  is 
e  =  x  —  xe,  then 

e  =  x  -  xe  =  f(x)  +  Bu{xe ,  t)  -  fc{xe,  t )  -  F(c(x),xe). 

Let  fc(xeit)  =  f(xe )  +  Bu(xe,  t)  —  F(c(xe),xe).  This  leads  to 

e  =  (f(x)  -  F(c(x),xe))  -  (f{xe)  -  F(c(xe),xe)).  (23) 
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As  an  indication  of  a  good  estimator,  we  want  the  error  to  be  asymptotically  stable,  so  that  e(t)  — >  0 
(or  xe(t)  — >  x(t))  as  t  — >  oo.  We  want  to  choose  a  function  F  which  will  satisfy  this  condition. 

For  a  linear  problem,  satisfying  the  stability  condition  is  fairly  straightforward,  as  described  in 
[2,  3].  The  dynamics  in  this  case  become  f[x)  —  Aqx.  the  measurement  becomes  c{x)  —  Cqx  and  we 
write  the  unknown  function  F{z,xe)  —  Lqz  —  LqCqx.  so  that  we  are  seeking  a  constant- valued  gain 
matrix  Lo-  The  problem  then  reduces  to  e  =  (To  —  LoCo)e.  This  is  asymptotically  stable  when  all 
the  eigenvalues  of  To  —  LqCq  have  negative  real  parts,  so  an  Lo  must  be  found  which  results  in  such 
eigenvalues.  Such  an  Lo  is  guaranteed  to  exist  if  the  pair  (To,  Co)  is  observable. 

An  ’’optimal”  choice  of  Lo  can  be  made  by  considering  this  problem  as  an  optimal  feedback 
control  problem.  Since  the  eigenvalues  of  a  matrix  are  the  same  as  those  of  its  transpose,  we  can 
change  the  problem  into  forcing  the  eigenvalues  of  (To  —  LoCo)T  —  Ty  —  Cq  Lq  to  have  negative 
real  parts.  However,  we  note  that  this  form  of  the  system  is  related  to  the  feedback  control  problem 
with  T  =  Aq,  B  —  Cq  and  L  —  Lq  : 

x  —  Ax  +  Bu 

\  roc  .  . 

J  —  -  J  [xT Mx  +  uTNuj  dt , 

with  a  control  u  —  —Lx.  The  optimal  feedback  gain  for  this  problem,  which  best  stabilizes  the 
system  given  the  cost  functional,  is  given  by  L  =  N  [BrL.  with  £  solving  the  Riccati  equation 

£T  +  Tt£  -  YiBN~1BtYi  +  M  =  0, 
or,  rewritten  in  terms  of  the  original  variables,  Lq  —  LCq  N  1  and 

£Tj  +  T0£  -  JV_1C0S  +  M  =  0. 

This  yields  the  ’’optimal”  state  estimation  gain  matrix  Lo  for  constructing  the  state  estimator 


xe  -  A0xe  +  Bu(xei  t)  +  L0{z-  C0xe) . 

Several  papers  in  the  literature  have  expanded  on  this  state  estimation  formula  to  include  non¬ 
linear  problems.  Thau  [10]  considers  systems  with  the  linear  and  nonlinear  parts  split,  such  as 

x  —  Aqx  +  g(x)  +  Bu ,  (24) 

where  g(x)  contains  only  second-order  and  higher  terms,  and  there  is  a  linear  measurement  function 
z  —  Cqx.  Thau’s  estimator  is  of  the  form 


xe  =  A0xe  +  g(xe)  +  Bu  +  L0  (z  -  C0xe )  , 

with  the  gain  matrix  L o  calculated  so  that  the  eigenvalues  of  the  linear  part  of  the  problem,  Aq— LqC\). 
have  negative  real  parts.  This  can  be  done  with  a  Riccati  equation  in  the  manner  described  earlier. 
This  leads  to  the  following  error  equation: 

e  =  (T0  -L0Co)e  +  g(x)  -  g(xe).  (25) 

Thus  if  g(x)  is  locally  Lipschitz,  the  estimator  will  locally  converge  asymptotically. 
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Kou,  Elliott  and  Tarn  [11]  consider  a  problem  with  nonlinear  dynamics  and  measurement  func¬ 
tion  and  present  a  condition  on  the  constant  gain  matrix  Lq  which  guarantees  that  the  estimator 

xe  =  f(xe)  +  Bu  +  Lq  (z  -  c(xe))  (26) 

is  asymptotically  stable  and  the  error  decreases  exponentially.  However,  it  is  often  very  difficult  or 
impossible  to  find  Lq  which  satisfies  this,  and  so  this  is  often  not  very  practical.  Mielczarski  [12]  and 
Hu  [13]  use  estimators  of  the  type  in  equation  (26),  finding  Lq  by  separating  the  dynamics  (as  well 
as  the  measurement)  into  linear  and  nonlinear  parts  like  Thau  did  in  equation  (24),  and  using  those 
linear  parts  to  find  the  gain  matrix.  They  do  this  separation  by  linearizing  f(x)  and  c(x)  about  the 
origin  or  some  other  expansion  point  to  obtain  matrices  Aq  and  Cq  for  the  linear  parts,  then  finding 
Lq  which  results  in  all  eigenvalues  of  Ho  —  LqCq  having  negative  real  parts. 

The  method  we  will  describe  here  extends  this  nonlinear  estimator  technique  further  in  a  some¬ 
what  different  direction.  Instead  of  using  a  completely  linearized  system  for  finding  the  gain  matrix 
To  5  we  will  use  a  state-dependent  Riccati  equation  to  solve  for  the  gain  from  the  nonlinear  system 
itself.  Rewriting  /  and  c  into  matrix  multiplication  form  as  f(x)  =  A(x)x  and  c(x)  =  C(x)x.  and 
replacing  F  by  F{c{x),xe)  =  L(xe)C(x)x ,  we  find  that  equation  (23)  becomes 

e  =  {A(x)  -  L(xe)C{x))x  -  ( A(xe )  -  L(xe)C{xe))xe. 

We  further  manipulate  the  equation  by  adding  and  subtracting  terms  to  change  it  into  a  form  more 
like  equation  (25): 


e  =  {A(x)  -  L(xe)C(x))x  -  ( A(xe )  -  L(xe)C(xe))xe  +  (1  -  l)(A(xe)  -  L(xe)C(xe))x 
=  ( A(xe )  -  L(xe)C(xe))e  +  {A{x)  -  A(xe)  -  L(xe)C(x)  +  L(xe)C(xe))x. 

We  seek  a  gain  L  such  that  e  =  ( A(xe )  —  L(xe)C(xe))e  is  asymptotically  stable,  and  assume  that 
the  remaining  term  is  small  (which  is  reasonable  if  x  is  small,  or  x  and  xe  are  close  and  A  and  C 
satisfy  certain  regularity  properties).  In  analogy  to  the  linear  case,  the  ’’optimal”  state-dependent 
estimator  gain  will  be  computed  from  a  state-dependent  Riccati  equation  as  outlined  below. 

As  in  the  linear  case,  we  consider  the  transpose  of  the  system,  AT(x)  —  CT(x)LT(x).  From 
there  we  set  up  the  related  feedback  control  problem  with  A  =  AL  .  B  =  CT  and  L  =  LT: 


x  = 

J  = 

u  = 

The  optimal  feedback  gain  is  now  given  by  L(x)  =  N~lB1  (x')H(.x).  with  E(.x)  solving  the  state- 
dependent  Riccati  equation 

T,(x)A(x)  +  AT  (x)T,(x)  —  H(x)R(x)7V_1RT(a:)H(x:)  +  M  =  0, 
or,  rewritten  in  terms  of  the  original  variables,  L(x)  =  T,(x)Ct (x)N~1  with  S(x)  satisfying 
Y1{x)At{x)  +  A{x)L{x)  -  T,(x)Ct (x)N~1C(x)T,(x)  +  M  =  0. 


A(x)x  +  B(x)u 
^  J  (xT Mx  +  utNuJ  dt 


—L(x)x. 
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This  fully  state-dependent  version  of  the  Riccati  equation  can  be  very  difficult  to  solve  due  to 
the  £ ( x )  C1 ' ( x )N~[  C (  j;  )  £  ( x )  term,  especially  if  the  measurement  function  is  complicated.  To  deal 
with  this,  we  will  make  a  simplification,  linearizing  the  measurement  for  the  purposes  of  finding  L(x). 
We  wish  to  keep  intact  as  much  of  the  nonlinear  nature  of  the  problem  as  possible,  though,  so  while 
we  remove  the  state-dependence  of  C(x ),  we  keep  A(x)  intact  in  the  Riccati  equation.  Having  done 
this,  we  can  use  the  SDRE  solution  method  described  in  Section  2  to  find  the  state  estimation  gain 

L(x)  =  E^JCgAT1  (27) 

from  the  state-dependent  Riccati  equation 

X(x)At(x)  +  A{x)^{x)  -  £(; x)Cq  N-'Co^ix)  +  M  =  0  (28) 

with  Co  —  (7(0).  The  nonlinear  dynamics  and  measurement  are  also  still  intact  in  the  main  part  of 
the  estimator, 


xe  -  f(xe)  +  Bu(xei  t)  +  L(xe)  (z  -  c(xe )) ,  (29) 

as  they  are  in  the  methods  of  Thau  and  others  described  above. 

5.  Application  to  Test  Problems 

5.1.  Simple  Example  System 

We  test  the  tracking  and  state  estimation  methods  in  Sections  3  and  4  first  on  a  simple  example 
problem  from  [30],  which  was  used  as  a  test  problem  for  the  nonlinear  control  method  survey  in  [1], 
The  nonlinear  control  system  is  given  by 


Xl 

0 

1  ’ 

Xl 

+ 

'  0  " 

X2 

T2 

X1 

0 

X2 

1 

in  a  factorized  form  appropriate  for  the  SDRE.  The  cost  functional  is  given  by 

J(x o,  u)  —  —  j  (xtQx  +  uT Ru'j  dt , 

or,  slightly  modified  for  the  tracking  problem, 

1  /  \ 

J{x o,  u)  =  -  [(y  -  r)TQ(y  -  r)  +  uT Ruj  dt. 


First  we  will  consider  just  the  feedback  tracking  control  problem,  with  no  state  estimation 
involved.  We  will  track  the  variable  y  —  x \ .  attempting  to  force  it  to  jump  up  from  0  to  0.5,  hold, 
and  then  jump  back  to  0  (as  shown  in  Figure  1).  The  weights  in  the  cost  function  will  be  set  to 
Q  —  10  and  R  —  1,  and  the  initial  condition,  of  course,  is  at  the  origin.  The  numerical  computation 
of  the  controls  and  solution  of  the  systems  are  done  with  MATLAB  code  written  by  the  authors, 
using  the  built-in  functions  ’’are”  for  solving  constant-valued  algebraic  Riccati  equations  and  ”ode45” 
for  solving  ODE  systems.  The  two-point  boundary  value  problem  for  the  tracking  variable  in  the 
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nonlinear  case  is  solved  with  a  final  time  of  Tj  =  15,  and  100  time  discretization  intervals.  The  first 
five  terms  of  the  SDRE  power  series  solution  are  used. 

The  results  of  the  tracking  control  problem  are  plotted  in  Figure  1  for  the  nonlinear  tracking 
control  described  in  Section  3  in  equations  (19)-(21),  as  well  as  a  linear  tracking  control  obtained 
by  linearizing  the  problem  and  then  using  the  standard  linear  techniques.  The  nonlinear  control 


Figure  1:  Comparison  of  feedback  tracking  controls  on  Example  1,  with  weight  Q=10. 

is  obviously  superior  here,  as  the  linear  control  drastically  overshoots  the  maximum  of  the  desired 
trajectory.  Raising  the  cost  function  weight  on  the  state  to  Q  =  100,  as  shown  in  Figure  2,  brings 
the  linearly  controlled  state  down  much  closer  to  the  nonlinear  case,  but  the  nonlinear  control  still 
produces  better  results. 

Next,  looking  at  a  control  problem  with  state  estimation  and  without  any  tracking  component, 
we  will  set  the  initial  condition  to  xq  =  [1,0]T  and  ask  the  control  to  force  the  system  to  0.  The 
initial  estimated  state  will  be  (xe)o  =  [1.3, 0]T,  slightly  off  of  the  actual  state,  forcing  the  estimator 
to  compensate.  The  estimation  will  be  based  on  the  nonlinear  measurement  z  —  c(x)  —  x\  +  x\X2- 
In  this  problem  we  will  set  the  cost  function  weights  to  Q  =  IOI2  ( I2  being  a  2  x  2  identity  matrix) 
and  R  =  1,  with  the  weights  in  the  state  estimator  gain  problem  being  M  =  I2  and  N  =  1.  In 
Figure  3  we  plot  the  actual  and  estimated  states  for  two  methods  of  state  estimation  described  in 
Section  4.  One  is  the  method  based  on  Thau’s  work,  which  is  the  nonlinear  estimator  in  equation 
(26)  using  a  linear  gain  found  with  a  constant-valued  Riccati  equation  (we  will  refer  to  this  as  the 
linear  SE  gain  control).  The  other  is  the  extension  of  this  which  is  described  in  equations  (27)-(29), 
a  nonlinear  estimator  using  a  nonlinear  gain  from  a  state-dependent  Riccati  equation.  The  control 
using  nonlinear  SE  gain  performs  slightly  better  than  the  linear  SE  gain  control  in  this  example, 
with  the  estimated  state  converging  to  the  actual  state  faster  and  the  state  moving  to  0  faster  as 
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Figure  2:  Comparison  of  feedback  tracking  controls  on  Example  1,  with  weight  Q=100. 


Figure  3:  Actual  and  estimated  states  for  feedback  controls/state  estimators  in  Example  1. 
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well.  A  control  found  by  completely  linearizing  the  problem  and  finding  a  linear  state  estimator  and 
control  was  tried  on  the  problem  but  failed  to  yield  state  convergence. 

State  estimation  is  now  added  to  the  tracking  control  problem,  using  the  same  desired  step 
function  as  in  Figures  1  and  2.  The  weights  in  this  case  are  Q  —  10,  R  —  1,  M  —  101-2  and  N  —  1, 
and  the  measurement  is  z  —  x\  +  x\X2  as  before.  The  initial  condition  is  xq  —  [0, 0]T  and  the  initial 
estimated  state  is  ( xe)o  —  [0.25, 0]T.  Figure  4  depicts  the  actual  and  estimated  states  x\  which 
result  from  the  feedback  control  combining  the  nonlinear  tracking  and  state  estimation  techniques. 
The  estimator  converges  very  nicely  to  the  actual  state  while  tracking  the  desired  trajectory  well. 


Figure  4:  Actual  and  estimated  states  for  nonlinear  tracking  control/state  estimator  in  Example  1. 

This  fully  nonlinear  control  is  plotted,  with  a  partially  nonlinear  control  using  the  nonlinear  tracking 
formula  and  the  linear  SE  gain,  and  with  the  fully  linear  control  found  from  the  linearized  system, 
in  Figure  5.  The  fully  nonlinear  control  performs  better  than  the  linear  SE  gain  control  in  forcing 
the  state  to  follow  the  desired  trajectory  (specifically  in  the  latter  half  of  the  time  period),  while  the 
fully  linear  control  is  far  less  effective. 

Finally,  we  consider  the  same  problem  as  in  Figures  4  and  5,  with  both  state  estimation  and 
tracking,  except  that  instead  of  a  deviation  in  the  initial  estimated  state  for  which  the  estimator  must 
compensate,  there  is  instead  added  random  noise  in  the  problem.  The  noise  consists  of  independent 
uniform  distributions  E\ (t)  and  £2 (i),  with  |e^|  <  0.1  (20%  of  the  maximum  desired  x\  trajectory), 
one  added  to  the  dynamics  and  the  other  to  the  measurement: 

{x  —  A(x)x  +  Bu  +  £1 

Z  —  X 1  +  X\X-2  +  62- 

In  Figure  6  we  compare  the  results  of  the  fully  nonlinear,  linear  SE  gain,  and  fully  linear  controls 
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Time 


Figure  5:  Comparison  of  tracking  controls/state  estimators  on  Example  1,  with  inaccurate  (xe)o- 

applied  to  this  problem.  The  fully  nonlinear  and  the  linear  SE  gain  controls  yield  very  similar  results 
here,  and,  as  in  Figure  5,  both  perform  much  better  than  the  fully  linear  control. 

5.2.  Flight  Dynamics  Simulation  Example 

The  second  example  to  which  we  apply  the  state  estimation  and  tracking  control  methods 
discussed  in  Sections  3  and  4  is  a  modified  version  of  the  flight  dynamics  example  from  [31]  (this  was 
also  used  as  test  problem  in  [1]).  The  system  is  given  by 

x  =  (A)  +  X2ANL)  x  +  Bu , 
where  the  matrices  Ag,  A^l  and  B  are  given  by: 


Ag 


Anl 


-0.0443 

1.1280 

0.0 

-0.0981  0.0 

-0.0490 

-2.5390 

1.0 

0.0 

-0.0854 

-0.0730 

19.3200 

-2.2700 

0.0 

22.6834 

0.0490 

2.5390 

0.0 

0.0 

0.0854 

0.0 

0.0 

0.0 

0.0 

20. C 

1 

-0.2317 

0.0 

0.0 

0.0 

0.0 

-1.2760 

-0.7922 

0.0 

0.0 

0.0206 

0.1020 

64.2940 

-13.9710 

0.0 

-5.4167 

1.2760 

0.7922 

0.0 

0.0 

-0.0206 

0.0 

0.0 

0.0 

0.0 

0.0 
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Figure  6:  Comparison  of  tracking  controls/state  estimators  on  Example  1,  with  noise. 
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The  cost  functional  to  be  minimized  is 

J{x o,  u)  =  2  J0  V-y  ~  r)TQ(y  ~r)  +  uTRuj  dt. 

when  in  tracking  problem  form.  The  state  variables  in  this  model  represent  the  flight  conditions  of 
the  aircraft:  x\  is  the  deviation  of  the  velocity  from  the  level  flight  trim  value  of  1  (lOOm/s)  (given 
in  units  of  (100m/s)),  X2  is  the  deviation  of  the  angle  of  attack  from  the  trim  value  of  4.2(7r/180) 
radians,  x 3  is  the  pitch  rate  in  rad/s,  X4  is  the  flight  path  angle  in  radians,  and  X5  is  the  deviation 
of  the  canard  deflection  angle  in  radians  from  the  trim  value,  which  is  not  given.  The  control  u  is 
the  input  canard  deflection  in  radians.  The  canards  are  control  flaps  which  can  deflect  downward 
by  up  to  90(7t/180)  radians.  (The  meaning  of  the  model  is  described  in  more  detail  in  [31].) 

In  the  first  test  we  consider  a  control  problem  with  state  estimation  but  no  tracking.  The 
feedback  control  will  attempt  to  force  the  state  variables  to  0  from  an  initially  large  angle  of  attack, 
given  by  an  initial  condition  of  xq  =  (0,  15(7t/180), 0, 0, 0)T.  However,  the  initial  estimated  state 
is  ( xe)o  =  (0, 20(7r/180), 0, 0, 0)T.  The  measurement  for  the  estimator  is  of  the  velocity  and  the 
canard  deflection,  so  that  z  =  c(x)  =  [x\,  X5]7  .  The  cost  functional  weights  are  given  by  Q  = 
and  R  =  100,  and  the  weights  for  the  estimator  gain  problem  are  M  =  IOOI5  and  N  =  I-2-  Figure  7 
depicts  the  actual  and  estimated  states  for  this  problem  using  the  linear  SE  gain  (Thau)  control  and 
the  nonlinear  SE  gain  control  found  with  an  SDRE  solution.  For  each  method  it  takes  some  time 
for  the  estimated  state  to  converge  to  the  actual  state,  and  for  both  to  be  forced  to  0,  but  they  both 
do  so  in  a  smooth  manner.  There  is  a  larger  oscillation  noticeable  in  the  linear  SE  gain  control, 


17 


0.35 


-  -  Actual  State:  Linear  SE  Gain 


Estimated  State:  Linear  SE  Gain 

-  Actual  State:  Nonlinear  SE  Gain 

-  -  Estimated  State:  Nonlinear  SE  Gain 


•0.1 1 - 1 - 1 - 1 

0  5  10  15 

Time  (s) 


Figure  7:  Actual  and  estimated  states  for  feedback  controls/state  estimators  in  Example  2. 


causing  slower  convergence.  On  the  other  hand,  with  the  fully  linear  state  estimation  and  control 
algorithm  applied,  the  system  remains  unstable  and  diverges  to  infinity. 

Next  we  will  consider  a  problem  with  tracking  but  no  state  estimation.  The  objective  is  to 
track  a  desired  flight  path  angle  X4,  increasing  it  gradually  from  0  up  to  45(7t/180),  holding,  and 
then  returning  to  0  (as  shown  in  Figure  8).  The  initial  condition  is  therefore  at  the  origin,  and 
the  cost  weights  are  Q  =  1  and  R  =  10.  The  results  are  plotted  in  Figure  8  for  the  nonlinear  and 
linear  tracking  controls.  Here  the  nonlinear  control  does  not  perform  substantially  better  than  the 
linear  control;  in  fact  the  linear  control  yields  better  results  at  the  top  of  the  ramp  up  to  45(7t/180)) 
though  in  the  late  time  period  the  linear  control  returns  to  0  more  slowly  than  the  nonlinear. 

In  considering  a  problem  with  both  tracking  and  state  estimation,  we  use  the  same  desired 
trajectory  as  before,  forcing  the  flight  path  angle  x,i  from  0  up  to  45(7t/180)  and  back.  Estimation  is 
added  using  a  measurement  of  the  velocity,  angle  of  attack,  and  canard  deflection:  z  =  [x\.  X2,  x$]T . 
The  weights  are  Q  =  1,  R  =  1,  M  =  IOO/5  and  N  =  I3.  The  actual  state  starts  at  the  origin,  as 
in  the  previous  tracking  problem,  but  the  estimated  state  starts  slightly  off  the  actual,  at  (xe)o  = 
(0, 0, 0, 5 (7r / 1 80) ,  0)T.  Figure  9  depicts  the  estimated  state  almost  converging  to  the  actual  state  by 
the  time  of  the  desired  x.\  increase,  and  remaining  close  to  the  actual  state  for  the  rest  of  the  time 
period.  In  Figure  10  we  plot  the  actual  state  when  controlled  using  our  fully  nonlinear  algorithm, 
as  well  as  when  using  the  linear  SE  gain  control,  and  the  fully  linear  control.  It  can  be  seen  that 
the  linear  control  overshoots  significantly  at  the  top  of  the  ascent  and  is  very  slow  to  return  to  0. 
The  other  two  methods  produce  virtually  identical  results,  the  difference  indiscernable  in  the  plots 
in  Figure  10. 

Finally,  we  alter  the  problem  used  in  Figures  9  and  10,  involving  both  state  estimation  and 
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Figure  8:  Comparison  of  feedback  tracking  controls  on  Example  2. 


Figure  9:  Actual  and  estimated  states  for  nonlinear  tracking  control/state  estimator  in  Example  2 


Figure  10:  Comparison  of  tracking  controls/state  estimators  on  Example  2,  with  inaccurate  (xe)o. 

tracking,  by  removing  the  deviation  in  the  initial  estimated  state  and  adding  random  noise  to  the 
problem.  The  independent  uniform  distributions  £i(t)  and  £2{t)1  with  |£fc|  <  2.25(7r/180)  (5%  of 
the  maximum  desired  flight  path  angle,  and  an  even  larger  percentage  of  the  actual  state  variables 
at  most  times),  are  added  to  the  dynamics  and  the  measurement  respectively: 

j  x  =  (A0  +  x2ANL)  x  +  Bu  +  £1 
\  z  =  [x1,x2,x5]T  +  £2. 

The  results  of  the  fully  nonlinear,  linear  SE  gain,  and  fully  linear  controls  applied  to  this  problem 
are  plotted  in  Figure  11,  where  it  can  be  seen  that  the  two  nonlinear  controls  are  again  very  similar, 
and  that  both  perform  better  than  the  linear  control  at  tracking  the  desired  trajectory  both  at  its 
maximum  and  as  it  returns  to  0. 

6.  Conclusions 

In  this  paper  we  have  considered  the  method  for  feedback  control  of  nonlinear  systems  using 
the  state-dependent  Riccati  equation  and  extended  it  into  a  feedback  tracking  control  method.  We 
have  also  modified  the  state  estimation  method  for  nonlinear  systems  established  in  the  literature  to 
include  a  nonlinear  gain  function  found  through  a  state-dependent  Riccati  equation.  Application 
of  these  new  techniques  to  two  selected  example  problems  provided  significant  control  authority  and 
distinct  advantages  in  comparison  with  the  linear  methods. 

As  mentioned  in  earlier  sections,  there  are  some  drawbacks  and  restrictions  to  the  new  techniques 
which  must  be  considered.  The  power  series  solution  of  the  SDRE  method  grows  inaccurate  when  the 
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Figure  11:  Comparison  of  tracking  controls/state  estimators  on  Example  2,  with  noise. 


states  move  farther  from  the  origin,  something  which  is  of  particular  concern  in  a  tracking  problem. 
In  solving  the  tracking  variable  equation  by  coupling  it  with  a  nominal  state  equation,  one  tacitly 
assumes  a  good  prediction  of  the  actual  state  behavior  for  the  control  to  be  effective.  There  are 
limitations  on  the  types  of  problems  to  which  the  SDRE  approach  can  be  applied  and  on  the  types 
of  signals  which  can  be  tracked,  and  the  SDRE  for  obtaining  the  nonlinear  state  estimation  gain 
uses  only  a  linearized  version  of  the  measurement  function.  While  these  are  nontrivial  factors  to 
consider,  the  methods  described  here  for  tracking  control  and  state  estimation  are  still  applicable  to 
a  large  class  of  important  control  problems,  and  their  performance  on  the  chosen  examples  provides 
improvement  (in  places  dramatic  improvement)  when  compared  to  previously  established  control 
techniques. 
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